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Lie  F' roC£.‘S5 insl  Institute  of  the  Uriiversitu  of  Souttiorn 
California  iriitiated  a research  proJi?ct  on  March  Ir  1973  uhulm' 
contract  NOOl 2'3-73-C- 1 507  from  the  Mavol  Undersea  Center  to  stuau 
various  schemes  of  intrafranie  codind  of  video  sidnals.  Tlie  object  of 
the  project  was  to  develop  an  efficient  techiiioue  of  t r ansmi  t L i rid 
video  sidnals  from  remotelu  piloted  vehicles  (RF'U)  to  dround  statioris> 
compatible  with  rattier  severe  Power  and  volume  restrictions  ori  tlie 
transmitter . 


The  primary  techniaues  that  have  been  explored  for  intraframe 
codind  are  transform»  linear  predictive  and  hybrid  t ransf o rm/ 1 inea r 
predictive  techiaues  The  hybrid  codind  system  whicfi  combines 

the  attractive  features  of  both  transform  arid  predictive  cocirid 

systems  has  been  studied  in  detail.  The  specific  transform  used  is 
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the  discrete  cosine  transform  . The  liybrid  cosine/DF'CM  system  nay 


be  implemented  to  operate  in  real  time  throudli  the  use  of  charde 


coupled  devices  and  conventional  DF'CM  Lechnioues  ■■^7. 


In  December  1974  r the  Imade  F'rocessind  Institute  eiufarked  on  the 
second  phase  of  the  study^  under  a separate  contract 
( N00123-75-C-1 192 ) from  the  Naval  Undersea  Centerr  to  study  the 
interframe  codind  of  RF'U  imades  and  to  extend  the  previously  developed 
codind  techniaues  to  combine  the  spatial  and  temporal  codind  of  imcdc 
seauences.  Codind  systems  based  upon  the  two  dimensional  cosine  and 
Fourier  transforms  combined  with  DF'CM  in  temporal  direction  were 
investidated  in  detail.  Results  were  presented  in  terms  of  cndirid 
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efficiency*  storesJe  rcoui  rcincTitc  * computational  comi- 1 o.;  i ty  > ?riU 
sensitivity  to  channel  noise  <5  . ( Reference  5 is  attached  to  th.r. 
final  report.)  F'erf ormance  of  the  hybrid  interf rame  coders  lias  ocen 
investisated  in  the  presence  of  channel  noise.  Results  show  that*  tor 
both  coder  implementatioris  studied*  minimal  imade  degradation  occutied 
for  a channel  error  probability  of  10  or  less. 

A series  of  experiments  were  performed  to  determine  the  tradeoff 
between  averade  bit  rate  per  frame  and  the  frame  repetition  rate  for  a 
fixed  channel  capacity.  The  measure  of  chanriel  capacity  used  was  thcj 
hit  rate  (BTR)*  which  is  defined  as  the  product  of  averade  pixel  bit 
rate  per  frame  end  frame  rate.  The  results  of  these  experiments  show 
that  reduced  frame  rates  coupled  with  cor respondindls  hidher  averade 
bit  rates  produce  smaller  MSE  for  the  individual  frames  coded.  This 
indicates  that  reductions  e::perienced  in  f r ame-to-f  r ame  cori-ela  tioris 
due  to  temporal  subsamplind  are  completely  coiiipensa  Led  for  by  tlie 
increased  number  of  bits  available  for  cod i rid.  However*  subjectively? 
reduced  frame  rates  tend  to  result  iri  Jerky  subject  iTiotion.  In  oi-der 
to  simulate  this  effect  on  actual  RF'd  data*  a new  64-frame  512  ::  512 
data  base  was  diditized  and  coded  usind  two-dimensional  cosine 
conibined  with  DF'CM  in  temporal  direction.  This  data  base  is  of 
tactical  sidnificance  since  it  represents  a low  altitude  flyover  of  a 
chemical  plant.  Computer  simulations  performed  on  this  data  base  have 
demonstrated  hidh  Quality  coded  imades  with  a 32J1  reduction  in  the 


averade  bit  rate 
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3.  Out  line  of  Futi.jre  St'J'dies 

In  this  section  fit  is  emeJiss i ::ed  that  furtJier  i itip roveinent  of  the 
hybrid  two  dimensional  t ransf o rm/DPCM  interframe  coder  is  possible.  A 
performance  evaluation  should  be  undertaken  to  e;:ercise  ttie  interfranie 
coder  usind  the  real  time  RF'*J  data  base.  Tliis  task  reeuires  in-deptfi 
investiSations  of  the  topics  mentioned  below. 


3.1  Channel  Noise  and  Error  Detection/Correction 


Simulations  to  evaluate  the  extent  of  imade  dedradation  due  to 
the  presence  of  channel  noise  have  previously  been  reported  for  the 
hybrid  two-dimensional  t ransf  o rm/DPCM  coders  <5>.  T!ie  Generally 
monotonical ly  non-decreasi' id  character  of  the  NMSE  curves  illustrates 
thatf  for  the  optimum  implementation  of  the  Piybrid  interframe  ccderr 
once  an  error  occurs?  its  rate  of  decay  is  slow  and  the  error"  tends  to 
propadate  in  the  temporal  direction.  From  previous  e;:perience  with 
intraframe  hybrid  transf orm/DF'CM  codersf  it  has  been  found  that  imade 
dedradations  due  to  the  inherent  DPCM  error  propadation  can  be  more 
rapidly  reduced  by  usind  smaller  than  optimal  weidfrtind  coefficients 
in  the  predictor  feedback  loop  <6>.  Thus?  tPie  predictor  is  operated 
in  a manner  wliich  is  suboPtimal  in  the  sense  of  minimicind  tPie  mean 
sauare  prediction  error.  However?  such  operation  Pias  tlie  advantede  of 
acceleratind  the  exponential  decay  in  the  contribution  of  transform 
coefficients  of  previously  scanned  pixels. 


This  concept  can  be  applied  to  the  hybrid  interframe  coder  since 
a first  order  linear  prediction  is  used  to  trarismit  transrorm 
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coefficient  differences  in  the  temporal  domain. 

It  has  been  o:;perimental  ly  verified  witti  Fisbrid  intrafianie  coders 
that  increases  in  imade  fidelity  can  be  achieved  by  post  processing  of 
the  encoded  imades  to  detect  and  then  correct  channel  induced  errors 
<7>.  F'rompt  correction  of  errors  detected  at  the  receiver  niinimi:;es 
dedradations  to  successive  frames  caused  by  error  p ropadat i on . 

The  proposed  detection  of  channel  induced  errors  is  a two  step 
process.  Initiallyr  the  assumption  is  made  that  differences  between 
decoded  transform  coefficients  for  successive  frames  are  small.  Thus» 
a threshold  approach  is  appropriate  where  the  threshold  is  selected 
such  that  if  the  computed  transform  coefficient  difference  between 
successive  frames  exceeds  this  threshold;  an  ei'i'or  is  considered 
likely.  The  second  stade  of  the  error  detection  process  seel'.s  to 
determine  if  an  observed  chande  which  exceeded  a diven  thiresFiold  is 
indeed  an  isolated  error  or  is  a valid  leadind  edde  of  a local 
increase  in  decoded  transform  coefficient  values.  This  determination 
is  based  on  the  exponen tial  decay  property  which  stiould  occur  dur-ind 
the  several  frames  followind  the  occurrence  of  an  isolated  eri-or. 

3.2  Bit  Transfer  Rate  (BTR) 

Hidh  Quality  RPf  imadery  in  a Jammable  environnient  depends  in 
part  on  the  frame  rate  emplovied.  In  many  RF'V  missions  such  as 
navidation  or  deneral  survei 1 lence ; the  frame  rate  can  be  as  low  as  1 
frame  per  second.  For  tardet  acuui s i t i on » hidher  frame  rates  oF  3 or 
4 fps  are  considered  adeouate  since  this  is  the  iiiaximum  rate  at  whicFi 
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a h'jiTian  operator  cari  as's i m i 1 ate  new  i ri f ormat  i on  . In  the  pi  opo-^eij 
si  mol  at  i ons » PTR  is  defined  as  the  product  of  avcrade  pi:;el  bit  rate 
per  franie  and  frame  rate  and  has  units  of  hits/pi;cel /secorid  t 

BTR  = ( bi  ts/pi:;el /f  rame  ) X (frames/sec) 


For  all  cases  ejiamined  with  previous  data  bases  it  was  shown  tliat 
reduced  frames  coupled  with  cor respondindls  hidher  averade  bit  rates 
result  in  smaller  normalised  MSE  values  for  the  individual  f ranios 
coded.  This  indicates  that  reduction  experienced  in  f rame- to-f rame 
correlations  due  to  temporal  subsamplind  are  completely  compensated 
for  by  the  increased  number  of  bits  available  for  codind.  Howeverf 
subjectively f reduced  frame  rates  tend  to  result  in  Jerky  subject 
motion.  This  is  most  apparent  for  rapidlu  movind  objects  in  the  field 
of  view>  and  is  of  lesser  conseauence  for  slowly  chandind  scenes. 
Intensive  verification  of  this  effect  based  on  RPU  imadery  remains  to 
be  investidated. 

3.3  Motion  Compensation 

A particular  aspect  of  the  RPV  data  base  seouence  is  that  it  is 
denerated  from  a movind  Platform.  Conseeuerit  1 y the  imades  in  the 
seouences  exhibit  f rame-to-f r ame  displacements  in  the  spatial  domain. 
Compensation  for  the  effects  of  motion  is  a problem  of  dreat  practical 
interest  as  f rame-to-f rame  displacements  reduces  the  correlatiori 
between  pixels  in  seauential  frames. 
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In  iJcrierelj  niovins!  ;3re3<i  iri  one  frcnie  uT  a television  eiiicial  are 
vers  similar  to  co r resrondi riS  areas  in  the  previous  frame  e;:cept  for  a 
linear  translation.  Thus  the  effect  of  motion  can  bs'  compensated  if 
the  speed  and  direction  of  motion  can  be  estimated  at  the  coder  <8;;  . 

Motion  compensation  usind  the  discrete  Fourier  transform  should 
be  investidated  for  the  hybrid  two-diiiiensional  transf orm/DF'CM  codei'. 
The  Fourier  transform  is  considered  because  of  its  phase  shift 
property?  a shift  in  the  spatial  domain  variables  results  in  a 
multiplication  of  the  Fourier  transform  of  the  un-shifted  imade  by  a 
phase  factor.  This  shiftind  property  is  expected  to  be  useful  iri 
detectind  and  compensatind  for  effects  of  motion  between  frames  since 
many  types  of  motion  such  as  Panned  motion  produce  sidnificant  chandes 
in  phase  components  and  small  chandes  in  amplitude  componerits.  Thus, 
compensation  for  Platform  induced  motion  may  be  implemented  directly 
in  the  array  of  phase  components  by  application  of  appropriate  phase 
correction  factors. 

3.4  Simplified  Encoders 

To  achieve  maximum  cod  i rid  performance  r tlie  hybrid  two-dimensional 
transf  orm/DF'CM  coder  is  desidned  to  be  adaptive  in  the  sense  that 
separate  statistics  andr  conseouently » separate  bit  allocations  are 
computed  for  each  subblock.  An  alternate  version  which  features  a 
much  simpler  implementation  is  a non-adaptive  or  simplified  coder, 
F'erformance  characteristics  of  the  simplified  coder  should  be  compared 
with  those  of  the  adaptive  coder.  A series  of  siniulations  should  be 
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pt» I'fo riTiC'd  ijsiri^  RF  V dcita  to  dote rm i rie  de^i rodat  j one  in  coder 
r-erforniance  resultirid  T i om  nori-adaetive  i iiiF-leiiientat  i oi  is  » e.a.»  uoo  of 
averasle  statistics  for  bit  ass  i annicrits  r scalinii  and  feedbacF  loop 
coefficients  and  use  of  simplified  auantizers.  The  objective  of  these 
simulations  is  the  design  of  an  intei-franie  coder  whose  per-formence 
closely  3Ppro;;imates  that  of  the  adaptive  coderr  but  whose  hardware 
implementation  is  considerably  simplified.  An  additional  topic  which 
should  be  investigated  is  variations  on  subblocK  partition  sice  used 
for  two  dimensional  transforms. 
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